Using a Rapid Incremental Solution Construction Approach to Maximise the Completeness and Correctness of a Set of Requirements for a System

While the problem of poor requirements engineering and management has been documented since the dawn of systems engineering the continual documentation and discussion of the problem requirements has not resulted in a practical solution to the problem. This paper discusses applying the Blackboard methodology used in Artificial Intelligence to the development of a suite of Computer Enhanced Systems Engineering (CESE) tools that may be able to alleviate much of the problem. The paper also describes conceptual prototypes of the tools that have already been developed, their effect on alleviating the problem of poor requirements, and some concepts for future tools in the suite. INTRODUCTION The systems and software development industry is characterized by a paradigm of project failure (Standish 1995). The situation has been described by Cobb’s Paradox (Voyages 1996), which stated “We know why projects fail, we know how to prevent their failure --so why do they still fail?” While the problem of poor requirements engineering and management has been repeatedly and widely discussed and documented for at least 10 years as a contributing cause of project failures (Hooks 1993; Kasser and Schermerhorn 1994; Standish 1995, Jacobs 1999; Carson 2001; etc.), the continual documentation and discussion of the problem of poor requirements engineering and management has not resulted in a practical solution to the problem. At the same time as systems engineering is suffering the problem of poor requirements engineering and management, Kemp et al. (2001) write that the knowledge management community needs to address several essential issues immediately, including: • A systems approach. Typically, knowledge management programs focus on narrow solutions. A holistic approach is needed. • An evolutionary process. There is no currently accepted process model that supports the continued evolution of knowledge management capabilities within the organisation. This paper describes an evolutionary process for the continued evolution of knowledge management capabilities and also proposes to tackle the problem of poor requirements engineering and management using an approach that is based on the • Blackboard methodology pioneered by Nii (1986), • Agent based approach of using a suite of software products for rapid software evolution based on a domain model was presented as a way to develop software that is quicker and less expensive to maintain (Glover and Bennett 1996). THE RISC APPROACH The approach is named Rapid Incremental Solution Construction (RISC)1. It is • an incremental multi-disciplinary approach based on the Cataract Methodology (Kasser 2003). • reductionist, namely to first provide a solution for a part of the problem, and then provide a solution for another part, and so on, until the entire problem (or at least a major part of it) is eventually solved. 1 The authors are very conscious of the risks involved in implementing partial solutions to a problem; hence the choice of acronym. Using a Rapid Incremental Solution Construction Approach to Maximise the Completeness and Correctness of a Set of Requirements for a System Page 2 DRAFT Submitted to INCOSE 2003 • An approach to solving the problem of poor requirements by developing a collaborative system that includes encapsulated tools, newly designed custom software, and human collaborators (Lander 1997). The RISC approach is to provide concept demonstrators of the functionality needed in the form of a suite of simple next generation prototype Computer Enhanced Systems Engineering (CESE) tools or software agents having similar user interfaces and each performing a small range of functions. Some of these tools are known as Prototype Educational Tools for Systems and Software (PETS) engineering Systems Engineering is focused on dealing with well-structured problems. However, the problem of poor requirements is complex and ill-structured problem, and hence not solvable by the traditional systems engineering process. One aspect of dealing with complex and ill-structured problems such as the problem of poor requirements is that an accumulation of knowledge in the system can often clarify the problem (Nii 1986). In our experience it also tends to suggest solutions. Nii suggests: “Consequently a knowledge engineer needs to engage in exploratory programming, Exploratory programming, as defined by Beau Sheil (1983) is a ‘conscious intertwining of system design and implementation.’ Waterman (1985) notes that expert system building is accomplished in developmental stages ranging from research prototype to filed prototype until a fielded systems is evolved. That is, expert systems are also developed incrementally.” This is the approach we have taken in developing the RISC suite as well as the individual tools. The evolutionary process is shown in Figure 1. The process begins with users interacting with information as shown in Figure 1a. The knowledge is in the users. Once the situational Use Case is monitored and understood, a software agent can be created to automate activities that are performed repeatedly as shown in Figure 1b. The evolution of the system can then be thought of as the migration of knowledge from the users to the agents. The RISC Suite of CESE tools consists of a suite of tools The following CESE tools have already been developed • The First Requirements Elucidation Demonstration (FRED) tool. • The Communications Requirements Evaluation & Assessment Prototype (CREAP). • The Operations Concept Harbinger (OCH). FRED is designed to prevent the production of poorly written individual requirements and elucidate existing requirements. Kasser (2002) proposed a partial solution to the problem of poorly written requirements by describing a simple prototype of a software tool that could improve the wording of requirements. When released in October 2002, this tool was named FRED. FRED performs a syntax check on the text of the requirement pointing out potential defects. In much the same way that a word processor grammar checker is used, it is up to the user to determine if the defect is there and how best to correct it. As more is learnt about defects in requirements wording, FRED can be upgraded to incorporate that knowledge. Future iterations of FRED might evolve into a product having similar features to the commercially available StyleWriter, the program that can analyse document and highlight all of its faults (at least according to the web site (StyleWriter 2002). FRED can be used in two roles: Figure 1a Start of Evolutionary Process for System Automation Figure 1b Software Agents Perform Some Automated Functions, the User the Others. Figure 1. Evolutionary System Development Using a Rapid Incremental Solution Construction Approach to Maximise the Completeness and Correctness of a Set of Requirements for a System Page 3 DRAFT Submitted to INCOSE 2003 • To provide immediate feedback to requirements writers that the document that they are compiling has poorly worded requirements. In this mode it helps prevent the publication of poorly written requirements. • To ensure that all existing requirements can be adequately tested, by identifying the individual requirements in a multiple-requirement paragraph of a Requirements Document. In this mode it facilitates the generation of a ‘requirements to test’ traceability matrix. CREAP is more ambitious. It is targeted for use in a specify-to-inventory situation. It assists the writers of requirements by directing them to only specify requirements that are feasible in a given concept of operations. Kasser and Cook (2002) described this tool, which employs a frame-based approach (Cook et al. 2001) to reason about the feasibility of the requirements. CREAP is a frame-based requirements engineering tool (FBRET) that integrates military communications domain expertise and requirements engineering practice. Specifically, it ensures that the set of equipment selected to meet the requirements imposed on a communications systems for a rapid force deployment will constitute a viable military information system. The OCH is the result of the application of information management technology to solving the problem of poorly written and articulated requirements as well as the effect of changing requirements for both single systems and Systems of Systems (Kasser et al., 2002). The OCH: • is a tool for capturing user needs without the explicit use of “text-mode requirements”; • may be thought of as a multimedia Operations Concept Document that also contains measures of effectiveness for each operational scenario; • helps to bridge the gap between the soft systems methodologies used in the early phases of the system development life cycle and the hard systems methodologies used in the construction of a system. A MULTI-DISCIPLINARY APPROACH The RISC approach to maximising the correctness and completeness of requirements uses techniques from several disciplines. These are: • The concept that a document can be considered as a report from a database. • Rapid prototyping of software. • The object-oriented concept of inheritance. • Expert systems and artificial intelligence. • Effective Configuration Management of software products and processes. • An adaptation of the Blackboard methodology used in Artificial intelligence. • The suite of agents approach. • The hierarchical perspective. • The need for simple tools. The concept that a document can be considered as a report from a database. For example, a Requirements Document can be considered as a printed view of a requirements database. Rapid prototyping of software. Software engineering with the ability to quickly create software has developed a methodology known as rapid prototyping. Rapid prototyping can be used to allow customers to see and use the user interface of a complex software program, or to develop and incrementally release small software packages such as FRED. Copies of programs like FRED can be distributed among the practitioner community and the resulting feedback can be examined and used to construct an upgraded version. The object-oriented concept of inheritance. Kasser (2000) stated, “To maximize the completeness of requirements and reduce the effect of non mission-specific missing requirements, both systems engineering and T&E must also ensure that system to be developed inherits the requirements for the specific type of product. For example, a low earth orbiting (LEO) spacecraft will inherit a set of generic requirements that have been refined from the experience gained in building these vehicles over the last fifty years. These requirements relate to the thermal, vacuum, and electromagnetic environment in space, launch, vibration, and salt spray, etc. Should the LEO spacecraft be a communications satellite, there would then be an additional set of generic requirements to be inherited.” Expert systems and artificial intelligence. Cook (1990, 1991, 1993) discusses a structured document generation and processing shell, which represents the earliest reported knowledge-based system for computeraided generation of equipment specifications. On completion of the specification elicitation, checking, and Using a Rapid Incremental Solution Construction Approach to Maximise the Completeness and Correctness of a Set of Requirements for a System Page 4 DRAFT Submitted to INCOSE 2003 Figure 2 The RISC Approach to Multiple Blackboards editing functions, the tool was able to process the model of the specification held in the frame structure in Prolog to produce a written document in MIL-STD-490A format. Kasser (1992) introduced ELMER, a simple expert system based on a state machine that parsed text and had different ways of reacting to a word match. Wu and Lee (1993) discuss the construction of an object-oriented expert system for designing local area networks based on functional requirements. Sutcliffe et al (1998) describe a system that reuses requirements knowledge through decision models and generic models. Effective Configuration Management of software products and processes is critical in conventional development. In rapid prototyping and RISC situations, configuration control is even more critical. Kasser (2002a) shows how configuration control can be used on both the product and process. An adaptation of the Blackboard methodology used in Artificial Intelligence. Nii (1986) states that the blackboard model is an approach not a tool. The term blackboard is derived from the architecture of the system in which the processing activity is organised to resemble a group of experts reading the contents of, and then writing responses on, a blackboard. The basic approach to problem solving in the blackboard framework is to divide the problem into loosely coupled subtasks. Geymayr and Ebecken (1995) extend the definition of a blackboard system to one that consists of several knowledge systems that communicate through a blackboard and are controlled by an inference mechanism. The suite of agents concept. The Agent based approach of using a suite of software products for rapid software evolution based on a domain model was presented as a way to develop software that is quicker and less expensive to maintain (Glover and Bennett 1996). Kasser (2000a) also described the concept of a suite of tools accessing a common expanded requirements database as an approach for tackling the problem of project failures due to poor requirements engineering and management. The hierarchical perspective. The problem of poor requirements can be viewed from a hierarchical perspective. Kasser and Schermerhorn’s (1994) requirements for writing requirements can be grouped into those that deal with • single requirements, and • sets of requirements. Thus the application of blackboard-based expert systems and via a suite of software agents to the problem of poor requirements is motivated by the same reasons as Weiss and Stetter (1992) had for using the Hierarchical Organised Parallel Expert System (HOPES) that provided a framework for working with realtime parallel expert systems, namely 1. A single blackboard may become a bottleneck in the process of solving the global problem. 2. The use of the multiple-blackboard architecture reduces the complexity of the problem into a number of simpler problems. 3. Managing the multiple-blackboard-based projects is easier than managing a big complex project because the management is spread over several smaller projects if the appropriate methodologies are used. In this case the Cataract Methodology (Kasser 2002a) manages the development of the PETS suite of CESE tools and the object-oriented concept of abstraction is used to focus on the specific defect(s) addressed by a tool. 4. The limited functionality of each agent or tool provides a way to develop software that is quicker to develop and modify and less expensive to maintain (Glover and Bennett 1996). In the RISC approach, the loosely coupled suite of blackboards becomes a suite of loosely coupled CESE tools as shown in Figure 2. Each tool operates on one or more elements in the hierarchy of defects in requirements. The users are the agents acting on the tool, and the tools are the agents operating on the central blackboard. The tools • perform useful functions in themselves, and Using a Rapid Incremental Solution Construction Approach to Maximise the Completeness and Correctness of a Set of Requirements for a System Page 5 DRAFT Submitted to INCOSE 2003 • serve as a discussion points about the scope and nature of the problem, and • serve as examples of functionality for tool vendors to implement. The knowledge application strategy is both top-down and bottom-up. The top-down strategy identifies the broad functionality of each of the specific tools, e.g. FRED elucidates poorly written requirements. The bottom-up strategy is driven by the results of applying the tool in the workplace. The first version of a tool uses knowledge provided by the constructors. As the tool is used, the knowledge transfers from the users to the developers in the form of positive comments in suggestions for upgrades and negative comments in the form of (constructive) criticism. This information finds its way to the central blackboard; in this case the PETS Suite Configuration Control Board (CCB). The CCB analyses the information and authorises changes in one or more of the tools as appropriate. For example, after some use of FRED, it was noticed that if there were a way to tell FRED to ignore a specific word in the next requirement, FRED’s functionality would be improved. This situation arose because of inexactness in today’s requirements. For example, consider the use of the word “and” in a requirement. A requirement such as "DADS shall display the combined number of apples and oranges" is a requirement to display a single total, hence the use of the word "and" is appropriate. On the other hand, a requirement such as "DADS shall display the number of apples and oranges" is two requirements, the first to display the number of apples, and the second to display the number of oranges. This is a defect in the requirement that complicates the building of traceability matrices and causes difficulties in the test and evaluation processes. In this instance the change is limited to a single tool. , The tool thus evolves under CCB control, becoming increasingly useful and over time, solves more and more parts of the problem. The suite approach allows for specific tools to be built or modified for testing hypotheses for improvements. Each hypothesis could be generated from a bottom-up or top-down source. Issues with the use of words. The military standards in the form of MIL-STD 490B and its replacement MIL-STD-961D (1995) now used as a ‘standard practice’ states that the word “shall” defines a requirement. However; there seem to be other opinions on this subject in the United States of America (USA). The Federal Aviation Agency (FAA) has a plain language initiative and published guidelines for writing documents in plain language. The document (FAA 2000) states “Shall” is one of those officious and obsolete words that has encumbered regulations and other documents for many years. The message that “shall” sends to the reader is, “this is boring material.” “Shall” is imprecise. It can indicate either an obligation or a prediction. Dropping “shall” is a major step in making your regulation more reader-friendly many agencies already use the word "must" to convey obligations with no adverse legal effects. You can avoid "shall" by substituting "must" to indicate an obligation or "will" to indicate that an action will occur in the future. Be careful to consider which meaning you intend to communicate to your readers. However, the word “shall” is precisely defined in the Random House dictionary and thus is not ambiguous at all. Moreover, “must” does not convey who is responsible for making the system comply as clearly as “shall”. The actual word that is used is immaterial, as long as everyone agrees to use the same word for a specific meaning. Until there is an agreement on vocabulary, requirements engineering cannot move off the baseline. Tools like FRED may help to clean up these situations by strongly suggesting the use of specific words. The blackboard system design process (McManus 1992) is shown in Figure 3. It is very similar to the rapid prototyping concept. The RISC approach based on using Concurrent Blackboard Systems is shown in Figure 4. It is very similar to, and based on, the Cataract methodology of Build planning (Kasser 2002b) for a single product and the RISC approach uses this methodology for managing the development of a suite of loosely coupled products that may be combined into a single multi-functional product sometime in the future. This is methodology is also the proven hardware approach of “build a little, test a little”. Integration is of the functionality and not necessarily of the tools. The software architecture is based on the arrangement and methodology of Kasser (1997) to facilitate the modularity. The need for simple tools. Once the usefulness of the functionality of these tools is proven, the subsequent generation of tools may or may not combine functionality. There is much to be said for avoiding the bloatware of current office automation products. Moreover, the current requirements engineering tools have a Using a Rapid Incremental Solution Construction Approach to Maximise the Completeness and Correctness of a Set of Requirements for a System Page 6 DRAFT Submitted to INCOSE 2003 Figure 5 Typical Display From an early MAX Prototype long learning curve, which sets a high threshold to be overcome when arguing for their adoption. Tools need to be simple to be used widely. The goal for the PETS suite of conceptual CESE tools is that they should be no more difficult to use than a slide rule and that each tool must help to improve the current situation.